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Abstract 

Wavelet coefficients are estimated recursively at progressively coarser scales recursively. As a 
j^ , result, the estimation is prone to multiplicative propagation of truncation errors due to quantization 

and round-off at each stage. Yet, the influence of this propagation on wavelet filter output has 
not been explored systematically. Through numerical error analysis of a simple, generic sub-band 
'^ . coding scheme with a half-band low pass finite impulse-response filter for down sampling, we show 



that truncation error in estimated wavelet filter coefficients can quickly reach unacceptable levels, 



C^^ ' and may render the results unreliable especially at coarser scales. 

PACS numbers: 02.70.Rr, 02.60.Gf, 02.30.Mv, 02.60.Cb 
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^ ■ INTRODUCTION and round-off ("truncation errors") in esti- 

mation in each scale. Sub-band coding (i.e., 

With the exception of Haar wavelets, breaking the signal into a number of differ- 

wavelet coefficients (e.g., in Coiflets or ent frequency bands and encoding each one 

Daubechies wavelets) are approximations to independently) and downsampling at each 

equations without closed-form solutions. Co- scale may determine the pattern of errors, 

efficients are recursively estimated at progres- Moreover, these truncation errors are not im- 

sively coarser scales, using the estimate on proved by sample size, and can dominate the 

one scale as the input to the next. Such variance of estimation. Yet, the propagation 

recursive schemes are prone to multiplica- of truncation errors and their influence on es- 

tive propagation of errors due to quantization timated wavelet coefficients have not been ex- 
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plored systematically. In this note, we report 
the results from an error analysis of a simple, 
generic sub-band coding scheme with a half- 
band low pass finite impulse-response (FIR) 
filter for down sampling. We demonstrate 
the sub-band coding scheme, and derive the 
bounds for signal-to-noise ratio of the filter 
output in Section „ and provide a numeri- 
cal analysis of truncation error propagation 
in Section - 



SIGNAL-TO-NOISE RATIO, SUB- 
BAND CODING, AND DOWNSAM- 
PLING 

Due to the dependencies between the vari- 
ables and the complexity of the expression, 
an exact formula for the signal-to-noise ratio 
(SNR) is intractable in general. However, an 
approximate expression (in decibels) can be 
given for the SNR 
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after a simple algebraic manipulation. The 
term in parenthesis consists of two approx- 
imately normally distributed random vari- 
ables (given a large sample size) with zero 
mean and a small variance, divided by a nor- 
mally distributed random variable with mean 
one and a small variance. Therefore, the dif- 
ference on the left-hand side will be relatively 
small with high probability. From now on, 
we assume the numerator and denominator 
in equation [T] can be replaced by its expecta- 
tion. 

Let fk be the downsampling filter. Then, 
for a fixed input 5* using a noise free filter of 
length N, the detail coefficients dyz take the 
form 
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where S and Af denote the power of the signal 

and noise. With small relative error, the ex- 

S 

pression in brackets can be replaced with =, 

where ^ denotes the mean. Assuming the re- 
spective coefficients of variation, c^ — 



and Cj\j- 
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are small (i.e. bounded by 



a a small constant e), the minimal relative 
error can be obtained as 



dyz - S2^y - 2^ II /ifc'S'2-J/-Efc = l2'=-ljfc 
ifce{l:iV} 

(3) 
When each of the filter coefficients fi is per- 
turbed by noise (ej ~ Af{0,Xi), where Aj is 
chosen to fix the input signal-to-noise ratio, 
SNR/) , the perturbed detail coefficient, dyz, 
takes the form 



At scale level z, the output signal to noise 
ratio (SNRo) is then given (in decibels) by 
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dyz - dyz - 2^ejfc [[ifij + %)^2'^y-j:Ui- 



kAi, i=l 



Yi n ^^^ n /v'^2-2/-E^.=i2'=-iife (6) 
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Assuming that E[SiSj] = 6ij, we can approx- imate the numerator as 

I 
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Note that the indices {ii,ik,i'k,k = are many low pass filters which are strictly 
2- ■ ■ z} are freely varying in the above equa- positive, and the gain can always be ad- 
tion. Since fk is a low pass filter with unity justed). Under these assumptions, and as- 
gain, we can assume Ylk fk =^- We also as- suming no exclusions due to truncation, it is 
sume that each of the f^ are positive (there easy to show that bounds for the numerator, 

E (c?^^), are 



max(0, 1 — 2 max /^ + min fk) < ¥,{d ) < 1 — 2 min fk + max fk (8) 
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10 logio nY 4^^ ^ 10 l°gio ^ - 10-^ logio 2 + logio(l - 2 mm fk + max /,) (9) 
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101ogioE(J]rfJj > 101ogioiV-2;logio2 + logio(max(0,l-2max/, + mm/fc)) (10) 



These inequalities will be true for real data weight. 

with a very small margin of error. How- Further note that by assumption, IE[ejej] 

ever, only inequality [9] is likely to carry much io~^^^^'^° — ——5ij. Therefore, in general. 



101ogioAr = E^(cij,, -4,)2 
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where fj is a factor that depends only on the 
filter coefficients fj under the above assump- 
tions. Equation [TTlfollows since \\S\\ = N, as 
S is assumed to be white noise. The expan- 
sion in equation [TT] involves terms weighting 
the higher Gaussian moments involving the 
perturbation of / in equation HI 

The calculation of the expectation of the 



denominator in equation [5] is quite compli- 
cated. However, if we further assume that 
the input SNR is sufficiently small so that 
the summand in equation [TT] is negligible, we 
only need to calculate f^. Under these as- 
sumptions, moments higher than the second 
in the matching random variable need not be 
considered, and 
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Equation [13] shows that we have all products of all ffiter coefficients of length 2z, but two 
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are excluded. The excluded ones represent of products, we again assume ^ ■ fj = 1. In 
the same scale, and are required to be eval- this case, 
uated at the same index. One remaining co- 
efficient is fixed so that both products of co- N'^z/2'' min |/| < fi < N^z/2'' max / 
efficients would contribute to the same detail 

coefficient. To obtain a bound on this sum and thus, 
I 

10 logio ^^71" "' ,p., < SNR,-10 logio N+W logio(l-2 min A+max /,) - 10 log^o z - log min f 

(14) 



when there is no truncation due to coeffi- A^ = 30 (downsampling filter FIRl [l|), and 
cients being out of bounds. chose the difference of the low pass filter from 

The two bounds (equations [9] and [14]) on the identity as our detail coefficients. Coef- 
the SNR posit that for each scale, the SNR ficients used in a typical estimation proce- 
will decrease linearly with slope one as a func- dure depend on two integers: m, the num- 
tion of the input SNR, and will fall off log- ber of times the data is downsampled prior 
arithmically per octave. That is, the actual to estimation, and k, the number of octaves 
data degrades at a rate of -1.6 decibels per used in the estimation. We used m = 7 and 
octave, and a linear fit to 10 logio ^ would /^ = 6 in numerical analyses of the overall 
yield a reduction of 0.92dB per octave. Ad- process for definiteness. The SNR for each 
ditional non-linear factors and other factors scale was also studied individually. For error 
not treated in this simple model are operative analysis, we perturbed the downsampling fil- 
in the numerical simulations. ter while systematically varying the standard 

deviation, Aj, of the noise e^ (equation H]) so 
that the input signal-to-noise ratio (SNR/) 
ranged from 380 to decibels (dB). For each 
input SNR, we chose 100 white noise time se- 
ries, perturbed the downsampling filter, and 
investigated the effect of the input SNR for 
each time series to the output SNR of the 
filter given by 



NUMERICAL ANALYSIS 

To study truncation errors numerically, we 
generated a unit-power white noise time se- 
ries {n = 2^^) (our results are qualitatively 
the same for different classes of time series), 
downsampled the time series by a factor of 
two using a FIR low pass filter of order 
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FIG. 1. The relation between input and output SNR for the perturbed wavelet filter over all 
wavelet scales (2^ — 2^, left panel) and for the coarse scale (2^, right panel) within double machine 
precision. 
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where dij is the i detail coefficient obtained 
at level j by subtracting the low pass ffitered ^ 

CO 

value from the coefficient prior to downsam- | 

o 

pling, and dijis the same coefficient obtained 
from the perturbed ffiter (cf. equation [5]). 

The SNR of the output (SNRo) was ap- 
proximately linearly related to that of the in- 
put (SNR/) (Figure [H left panel) with the 
equation 
C (SNRo) = -25.8 + 0.994 xO(SNRi) (16) 



This relation shows that on average, a 10~^ 
error in the estimated filter coefficients re- 
sults in a 10~^ error in the output. It should 
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Input SNR (dB) 



FIG. 2. Degradation of output SNR as a func- 
tion of scale. Progressively lower traces depict 
the relation for progressively coarser scales, from 
213 to 22. 




TABLE I. Growing intercept of the relation be- 
tween input and output SNR with coarser scales. 
See also Figure [3l 

Scale (2") Intercept 



100 200 300 

Input SNR (dB) 



FIG. 3. The linear relation between input and 
output SNR for scales 2^ to 2^3 in the 50 - 270 
dB input SNR range. Lower traces show the 
relation for coarser scales. 

be noted, however, that this is a conserva- 
tive estimate for the overall output error. In 
fact, stringent constraints may be necessary 
to attain sufficient accuracy at coarser scales 
because of the degradation of output SNR 
(Figure E]). 

Complete relation between C(SNRo) and 
^(SNR/) for the range of double machine 
precision (Figure [H right panel) shows that 
between approximately 50 to 270 dB input 
SNR, output SNR is approximately a linear 
function (equation [TB]) . In contrast, output 
SNR is flat when the input SNR is greater 
than 270 dB (where less than one bit of noise 
is added), and accelerates to — oo from dB 
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when the input SNR is below 50 dB. More- 
over, although the output SNR is linearly re- 
lated to that of the input with a slope of 1 
for all scales ranging from k = 2"^ to k = 2^ 
within the range of 50 — 270 dB input SNR 
(Figure [31 cf. equation [T6|) . the intercept of 
this relation degrades at coarser scales (Ta- 
ble H]). Therefore, coarser scales are dispro- 
portionally affected. These results highlight 
that truncation errors in wavelet coefficients 
can quickly reach to unacceptable levels. For 
example, suppose that one requires the noise 
in the output of the coarse scale wavelet co- 
efficients to be less than 1% of the output 
power. In this case, the noise power in the 
filter coefficients must be 100 dB, or the fil- 
ter coefficients must produce errors less than 
approximately 5 x 10 ~^. 



SNR = 320.9 - 0.66 x (Octave) 
R2 . O.E 




of the A^ elements inverted, wavelet filter- 
ing and sub-band coding schemes can re- 
quire 30^°§2^ calculations. This difference 
can be significant at moderate input SNRs 
since repeated truncation errors may result 
in a catastrophic loss of precision. 

CONCLUSION 



FIG. 4. An error analysis for the FFT of size 2®. 
The output SNR is over 300 dB, and the reduc- 
tion in precision is about 0.85 for each additional 
octave. Vertical bars show 95% conhdence inter- 
vals. 

To put this error amplification in per- 
spective, consider simple filtering via Fast 
Fourier Transform (FFT) of up to size 2^°. 
In this case, SNR of the signal degrades only 
by about 20 dB from its maximal value of 
306 - 325 dB (Figure H]). This relatively 
small loss indicates that in practice, mem- 
ory would be exhausted before the trunca- 
tion error becomes a problem. In contrast, 
the sub-band coding scheme reported here 
(thus, most wavelet filters) require a signif- 
icant length convolution (A^ = 30) at each 
stage of the calculation. Thus, whereas the 
FFT requires log2 A^ multiplications for each 



Our results show that truncation error in 
the wavelet filter coefficients due to quanti- 
zation and round-off may amplify due to the 
the multiplicative propagation, and can reach 
substantial levels. In cases where the relative 
error in the wavelet coefficients is too large, 
the wavelet tree (k) grows too deep, or the 
wavelet filter (A^) is too long, this propaga- 
tion of truncation errors may render the re- 
sults unreliable especially at coarser scales. 
Note that although the sub-band analysis 
presented here is not identical to downsam- 
pling using Coiflets or Daubechies filters |2|, 
small errors in the coefficients will still prop- 
agate in the latter case because the prop- 
agation depends strongly on the length of 
the filter applied prior to the downsampling. 
We expect errors with relatively large D15 
(A^ = 30 coefficient Daubechies) filter to be 
comparable to the case presented here. 



Imike@cns.bu.edul 



8 



' cotan@partners.org [2] I. Daubechies, Ten Lectures on Wavelets 

•f Corresponding Author. CVLab, Spaulding (CBMS-NSF Regional Conference Series in 

Rehabilitation Hospital, 125 Nashua Street, Applied Mathematics), The Art of Computer 

Boston, MA 02114 Programming (SIAM: Society for Industrial 

[1] T. Short, U. Ligges, S. Schnackenberg, H.- and Applied Mathematics, 1982), 1st ed. 
W. Borchers, and S. Krey, Signal processing 
toolbox for R (2011). 



